1. Field of the Invention
The present invention relates to gesture based modeling.
2. Description of Prior Art
Object-oriented programming languages provide many technical qualities but, more importantly, object-oriented languages and object-oriented development in general also provide a conceptual framework for understanding and modeling. This conceptual framework provides abstraction mechanisms for modeling, such as concepts (classes), phenomena (objects), and relations between these (inheritance, association, dependency) that allow developers to describe what their system is all about and to formulate solutions on a higher level of abstraction than program code.
In practice, developers build models at several abstraction levels. The program code can be seen as an executable model, but for purposes such as analysis, specification, documentation, and communication, models visualized graphically in the form of diagrams are often used. The Unified Modeling Language, UML [Rumbaugh, J., Jacobson, I., & Booch, G. (1999). The Unified Modeling Language Reference Manual. Addison Wesley] is a prominent example of a graphical notation that is used for such purposes. The Unified Modeling Language is a general-purpose visual modeling language that is used to specify visualize, construct, and document the artifacts of a software system.
Models have two major aspects: semantic information (semantics) and visual presentation (notation). Semantic model elements carry the meaning of the model, and they are used for code generation, validity checking, complexity metrics etc. The semantic information is often for simplicity called the model. The visual presentation shows semantic information in a form that can be seen, browsed, and edited by humans. Presentation elements carry the visual presentation of the model—that is, they show it in a form directly apprehensible by humans. Presentation elements are shown on diagrams. The UML is formally defined using a metamodel—a model of the constructs in UML. The UML metamodel defines how a definite model may be constructed, i.e., it defines the set of all legal UML models.
UML models are often used in object-oriented software development. Simplistically, object-oriented software development can be viewed as mapping a set of real-world phenomena and concepts to corresponding objects and classes. The set of real-world phenomena and concepts is called the referent system and the corresponding computerized system is called the model system.
Mapping a referent system to a model system is called modeling. Modeling is often iterative, and it is thus important to be able to discuss a model system in terms of the referent system. We call this reverse process interpretation. Modeling is concerned with expressing an understanding of a referent system in a fluent, formal, and complete way, for which usable and formal notations, such as diagramming techniques, are crucial. Interpretation is concerned with understanding a model system in terms of the referent system. For doing this, it is important that central concepts in a model system are understandable in the referent system.
Neither understanding the referent system nor building a model system is unproblematic. Understanding the referent system is within the domain of user-oriented disciplines such as ethnography and participatory design. The ethnographic perspectiv on system development is concerned with basing system design on actual work performed within the referent system. The participatory design perspective on system design is concerned with designing systems in active collaboration with potential users. Both perspectives produce knowledge of the referent system. Finally, building the model system is within the domain of object-oriented software engineering. Software engineering is concerned with building robust, reliable, and correct systems. In iterative development, then, co-ordination, communication, and collaboration between different perspectives and individual developers is crucial. Thus, when considering tool support for a system development process, tool support for the modeling and the interpretation processes is important.
In iterative development, modeling and interpretation are interleaved. When different competencies work together, it is crucial that both processes are, to a certain extent, understood by all involved parties. For example, during user involvement it is both important that the developers understand the work of the users (i.e., the referent system) and that the users are able to react on and help design the application (i.e., the model system.)
The so-called Computer-Aided Software Engineering tools (CASE tools) are diagram editors that provide tool support for modeling. These tools let a user design a system using a graphical design notation and usually generate at least code skeletons or a code framework. CASE tools, among other things, help users to create, edit, and layout diagrams, to perform syntactic and semantic checks of models, to simulate and test models, to share diagrams between and combine diagrams from several users, to generate code or code skeletons from models (forward engineering) and generate diagram or diagram sketches from code (reverse and round-trip engineering), and to produce documentation based on models and diagrams. Many CASE tools are based on UML, i.e., they allow users to create UML models and diagrams.
Examples of commercially available CASE tools are Rational Rose® and Microgold With-Class™.
But even though CASE tools offer these many attractive features, they are in practice not widely and frequently used. CASE tools clearly support design and implementation phases, but have less support for the initial phases, when the focus is on understanding the problem domain and on modeling the system supporting the problem domain [Jarzabek, S. & Huang, R. (1998) The Case for User-Centered CASE Tools. Communications of the ACM, 41 (8)].
In practice, CASE tools are supplemented with whiteboards in the creative phases of development. The most appealing aspects of whiteboards are their ease of use and their flexibility. Whiteboards require no special skills, they do not hamper the creativity of the user, and they can be used for a variety of tasks. Their many advantages aside, for most development projects whiteboards are not sufficient, as they do not support, for example, advanced editing and loading and saving a diagram.
It would thus be desirable if CASE tools had support for intuition, flexibility, and collaboration, giving a more direct, less complex user-interface, while preserving the current support for more technical aspects such as implementation, testing, and general software engineering issues.
As user studies have shown [Damm, C. H., Hansen, K. M., & Thomsen, M. (2000). Tool Support for Cooperative Object-Oriented Design: Gesture Based Modeling on an Electronic Whiteboard. In Proceedings of Computer Human Interaction (CHI'2000), The Hague, The Netherlands], CASE tools are primarily used for code generation, reverse engineering, and documentation, whereas whiteboards are used for collaborative modeling and idea generation. This mix causes a number of problems: Whereas whiteboards are ideal for quickly expressing ideas collaboratively and individually, they are far from ideal for editing diagrams etc. This means that in all the user studies, drawings have been transferred from whiteboards to CASE tools and from CASE tools back to whiteboards. The conflicting advantages and disadvantages of whiteboards and CASE tools thus lead to frustrating and time consuming switches between the two technologies.
Electronic whiteboards have been used as a computational extension of traditional whiteboards. A goal in systems running on electronic whiteboards has often been to preserve desirable characteristics of whiteboards such as lightweight interaction and informality of drawings.
From a project at the University of California, it is known to combine the versatility of an electronic whiteboard with object-oriented modeling. An electronic whiteboard is coupled to a sketch interpreter in order to recognize a variety of drawn gestures on the whiteboard as classes and relations in UML static structure diagrams. However, though the method facilitates modeling of UML diagrams, the method does not imply the variety of features and modeling possibilities as known from CASE tools.
It is the purpose of the invention to provide a method for modeling employing the ease and flexibility of user-friendly input and output devices, such as whiteboards where the modeling is dependent on predetermined metamodels, for example to secure compatibility with external computer programs, preferably CASE tools.